Gradient free parameter estimation for hidden Markov models with intractable likelihoods

In this article we focus on Maximum Likelihood estimation (MLE) for the static model parameters of hidden Markov models (HMMs). We will consider the case where one cannot or does not want to compute the conditional likelihood density of the observation given the hidden state because of increased computational complexity or analytical intractability. Instead we will assume that one may obtain samples from this conditional likelihood and hence use approximate Bayesian computation (ABC) approximations of the original HMM. Although these ABC approximations will induce a bias, this can be controlled to arbitrary precision via a positive parameter , so that the bias decreases with decreasing . We first establish that when using an ABC approximation of the HMM for a fixed batch of data, then the bias of the resulting log- marginal likelihood and its gradient is no worse than O(n), where n is the total number of data-points. Therefore, when using gradient methods to perform MLE for the ABC approximation of the HMM, one may expect parameter estimates of reasonable accuracy. To compute an estimate of the unknown and fixed model parameters, we propose a gradient approach based on simultaneous perturbation stochastic approximation (SPSA) and Sequential Monte Carlo (SMC) for the ABC approximation of the HMM. The performance of this method is illustrated using two numerical examples

Linear variance bounds for particle approximations of time-homogeneous Feynman-Kac formulae

10.1016/j.spa.2012.02.002Stochastic Processes and their Applications12241840-1865STOP

On the convergence of adaptive sequential Monte Carlo methods

In several implementations of Sequential Monte Carlo (SMC) methods it is natural and important, in terms of algorithmic efficiency, to exploit the information of the history of the samples to optimally tune their subsequent propagations. In this article we provide a carefully formulated asymptotic theory for a class of such adaptive SMC methods. The theoretical framework developed here will cover, under assumptions, several commonly used SMC algorithms [Chopin, Biometrika 89 (2002) 539–551; Jasra et al., Scand. J. Stat. 38 (2011) 1–22; Schäfer and Chopin, Stat. Comput. 23 (2013) 163–184]. There are only limited results about the theoretical underpinning of such adaptive methods: we will bridge this gap by providing a weak law of large numbers (WLLN) and a central limit theorem (CLT) for some of these algorithms. The latter seems to be the first result of its kind in the literature and provides a formal justification of algorithms used in many real data contexts [Jasra et al. (2011); Schäfer and Chopin (2013)]. We establish that for a general class of adaptive SMC algorithms [Chopin (2002)], the asymptotic variance of the estimators from the adaptive SMC method is identical to a “limiting” SMC algorithm which uses ideal proposal kernels. Our results are supported by application on a complex high-dimensional posterior distribution associated with the Navier–Stokes model, where adapting high-dimensional parameters of the proposal kernels is critical for the efficiency of the algorithm

Sequential Monte Carlo methods for Bayesian elliptic inverse problems

In this article, we consider a Bayesian inverse problem associated to elliptic partial differential equations in two and three dimensions. This class of inverse problems is important in applications such as hydrology, but the complexity of the link function between unknown field and measurements can make it difficult to draw inference from the associated posterior. We prove that for this inverse problem a basic sequential Monte Carlo (SMC) method has a Monte Carlo rate of convergence with constants which are independent of the dimension of the discretization of the problem; indeed convergence of the SMC method is established in a function space setting. We also develop an enhancement of the SMC methods for inverse problems which were introduced in Kantas et al. (SIAM/ASA J Uncertain Quantif 2:464–489, 2014); the enhancement is designed to deal with the additional complexity of this elliptic inverse problem. The efficacy of the methodology and its desirable theoretical properties, are demonstrated for numerical examples in both two and three dimensions

Error bounds and normalising constants for sequential monte carlo samplers in high dimensions

In this paper we develop a collection of results associated to the analysis of the sequential Monte Carlo (SMC) samplers algorithm, in the context of high-dimensional independent and identically distributed target probabilities. TheSMCsamplers algorithm can be designed to sample from a single probability distribution, using Monte Carlo to approximate expectations with respect to this law. Given a target density in d dimensions our results are concerned with d while the number of Monte Carlo samples, N, remains fixed. We deduce an explicit bound on the Monte-Carlo error for estimates derived using theSMCsampler and the exact asymptotic relative L2-error of the estimate of the normalising constant associated to the target. We also establish marginal propagation of chaos properties of the algorithm. These results are deduced when the cost of the algorithm is O(Nd2). © Applied Probability Trust 2014

On large lag smoothing for hidden markov models

In this article we consider the smoothing problem for hidden Markov models (HMM). Given a hidden Markov chain $\{X_n\}_{n\geq 0}$ and observations $\{Y_n\}_{n\geq 0}$, our objective is to compute $\mathbb{E}[\varphi(X_0,\dots,X_k)|y_{0},\dots,y_n]$ for some real-valued, integrable functional $\varphi$ and $k$ fixed, $k \ll n$ and for some realisation $(y_0,\dots,y_n)$ of $(Y_0,\dots,Y_n)$. We introduce a novel application of the multilevel Monte Carlo (MLMC) method with a coupling based on the Knothe-Rosenblatt rearrangement. We prove that this method can approximate the afore-mentioned quantity with a mean square error (MSE) of $\mathcal{O}(\epsilon^2)$, for arbitrary $\epsilon>0$ with a cost of $\mathcal{O}(\epsilon^{-2})$. This is in contrast to the same direct Monte Carlo method, which requires a cost of $\mathcal{O}(n\epsilon^{-2})$ for the same MSE. The approach we suggest is, in general, not possible to implement, so the optimal transport methodology of \cite{span, parno} is used, which directly approximates our strategy. We show that our theoretical improvements are achieved, even under approximation, in several numerical examples

Asymptotic behaviour of the posterior distribution in approximate Bayesian computation

Approximate Bayesian computation (ABC) is a popular technique for approximating likelihoods and is often used in parameter estimation when the likelihood functions are analytically intractable. In the context of Hidden Markov Models (HMMs), we analyse the asymptotic behaviour of the posterior distribution in ABC based Bayesian parameter estimation. In particular we show that Bernstein-von Mises type results still hold but that the resulting posterior is biased in the sense that it concentrates around a point in parameter space that differs from the true parameter value. Furthermore we obtain precise rates for the size of this bias with respect to a natural accuracy parameter of the ABC method. Finally we discuss, via a numerical example, the implications of our results for the practical implementation of ABC

Population-Based Reversible Jump Markov Chain Monte Carlo

In this paper we present an extension of population-based Markov chain Monte Carlo (MCMC) to the trans-dimensional case. One of the main challenges in MCMC-based inference is that of simulating from high and trans-dimensional target measures. In such cases, MCMC methods may not adequately traverse the support of the target; the simulation results will be unreliable. We develop population methods to deal with such problems, and give a result proving the uniform ergodicity of these population algorithms, under mild assumptions. This result is used to demonstrate the superiority, in terms of convergence rate, of a population transition kernel over a reversible jump sampler for a Bayesian variable selection problem. We also give an example of a population algorithm for a Bayesian multivariate mixture model with an unknown number of components. This is applied to gene expression data of 1000 data points in six dimensions and it is demonstrated that our algorithm out performs some competing Markov chain samplers

Parameter Estimation for Hidden Markov models with Intractable Likelihoods

In this talk I consider sequential Monte Carlo (SMC) methods for hidden Markov models. In the scenario for which the conditional density of the observations given the latent state is intractable we give a simple ABC approximation of the model along with some basic SMC algorithms for sampling from the associated filtering distribution. Then, we consider the problem of smoothing, given access to a batch data set. We present a simulation technique which combines forward only smoothing (Del Moral et al, 2011) and particle Markov chain Monte Carlo (Andrieu et al 2010), for an algorithm which scales linearly in the number of particles